{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**基于Jupyter的特征工程笔记本3: 特征降维**  \n",
    "*作者: 陈颖祥，杨子唅*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Dimension-Reduction-特征降维\" data-toc-modified-id=\"Dimension-Reduction-特征降维-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Dimension Reduction 特征降维</a></span><ul class=\"toc-item\"><li><span><a href=\"#Unsupervised-Methods-非监督方法\" data-toc-modified-id=\"Unsupervised-Methods-非监督方法-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span>Unsupervised Methods 非监督方法</a></span><ul class=\"toc-item\"><li><span><a href=\"#PCA-(Principal-Components-Analysis)-主成分分析\" data-toc-modified-id=\"PCA-(Principal-Components-Analysis)-主成分分析-1.1.1\"><span class=\"toc-item-num\">1.1.1&nbsp;&nbsp;</span>PCA (Principal Components Analysis) 主成分分析</a></span></li></ul></li><li><span><a href=\"#Supervised-Methods-监督方法\" data-toc-modified-id=\"Supervised-Methods-监督方法-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span>Supervised Methods 监督方法</a></span><ul class=\"toc-item\"><li><span><a href=\"#LDA-(Linear-Discriminant-Analysis)-线性判别分析\" data-toc-modified-id=\"LDA-(Linear-Discriminant-Analysis)-线性判别分析-1.2.1\"><span class=\"toc-item-num\">1.2.1&nbsp;&nbsp;</span>LDA (Linear Discriminant Analysis) 线性判别分析</a></span></li></ul></li></ul></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Reference**\n",
    "- https://scikit-learn.org/stable/modules/decomposition.html#principal-component-analysis-pca\n",
    "- https://sebastianraschka.com/faq/docs/lda-vs-pca.html\n",
    "- https://en.wikipedia.org/wiki/Linear_discriminant_analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dimension Reduction 特征降维"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "经过数据预处理和特征选择，我们已经生成了一个很好的特征子集。但是有时该子集可能仍然包含过多特征，导致需要花费太多的计算能力用以训练模型。在这种情况下，我们可以使用降维技术进一步压缩特征子集。但这可能会降低模型性能。\n",
    "\n",
    "同时，如果我们没有太多时间进行特征选择，我们也可以在数据预处理之后直接应用降维方法。我们可以使用降维算法来压缩原始特征空间直接生成特征子集。\n",
    "\n",
    "具体来说，我们将分别介绍PCA和LDA（线性判别分析）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unsupervised Methods 非监督方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PCA (Principal Components Analysis) 主成分分析"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "主成分分析（PCA）是一种无监督机器学习模型，其目标为利用线性变换将原始特征投影为一系列线性不相关的单位向量，而同时保留尽可能多的信息（方差）。您可以从我们在Github中编写的[repo](https://github.com/YC-Coder-Chen/Unsupervised-Notes/blob/master/PCA.md)中查看更多数学细节。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-03-30T03:37:20.574170Z",
     "start_time": "2020-03-30T03:37:18.879555Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.decomposition import PCA\n",
    "\n",
    "# 直接载入数据集\n",
    "from sklearn.datasets import fetch_california_housing\n",
    "dataset = fetch_california_housing()\n",
    "X, y = dataset.data, dataset.target # 利用 california_housing 数据集来演示\n",
    "\n",
    "# 选择前15000个观测点作为训练集\n",
    "# 剩下的作为测试集\n",
    "train_set = X[0:15000,:]\n",
    "test_set = X[15000:,]\n",
    "train_y = y[0:15000]\n",
    "\n",
    "# 在使用主成分分析前，我们需要先对变量进行缩放操作，否则PCA将会赋予高尺度的特征过多的权重\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "model = StandardScaler()\n",
    "model.fit(train_set) \n",
    "standardized_train = model.transform(train_set)\n",
    "standardized_test = model.transform(test_set)\n",
    "\n",
    "# 开始压缩特征\n",
    "compressor = PCA(n_components=0.9) \n",
    "# 将n_components设置为0.9 =>\n",
    "# 即要求我们从所有主成分中选取的输出主成分至少能保留原特征中90%的方差\n",
    "# 我们也可以通过设置n_components参数为整数直接控制输出的变量数目\n",
    "\n",
    "compressor.fit(standardized_train) # 在训练集上训练\n",
    "transformed_trainset = compressor.transform(standardized_train) # 转换训练集 (20000,5)\n",
    "# 即我们从8个主成分中选取了前5个主成分，而这前5个主成分可以保证保留原特征中90%的方差\n",
    "\n",
    "transformed_testset = compressor.transform(standardized_test) # 转换测试集\n",
    "assert transformed_trainset.shape[1] == transformed_testset.shape[1] \n",
    "# 转换后训练集和测试集有相同的特征数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-03-30T03:37:21.146354Z",
     "start_time": "2020-03-30T03:37:20.576580Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEHCAYAAACumTGlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XeYVdW5x/Hvy9B7r9KrSHdoEXUQUaxXsSvYRY0libmJJmo0Br2J3aioKCgqalCxNywgoHSRptJ77wMM09/7x94TcTwwZ2DOnCm/z/PMwx7OOue8bGb27+y191rL3B0REZEy8S5ARESKBgWCiIgACgQREQkpEEREBFAgiIhISIEgIiKAAkFEREIKBBERARQIIiISKhvvAvKjbt263qJFi3iXISJSrMyZM2ebu9fLq12xCoQWLVowe/bseJchIlKsmNnqaNqpy0hERAAFgoiIhBQIIiICKBBERCSkQBAREUCBICIiIQWCiEgRl5GVTUZWdszfJ6aBYGajzGyamd11kMdbmtlHZjbFzB6JZS0iIsVNRlY2b8xcw0mPTOI/s9bG/P1iFghmNhhIcPe+QCszaxuh2b+Af7j78cBRZpYUq3pERIqL9MxsXpuxhqSHJnHH+AXUqlyelnWrxPx9YzlSOQkYF25PAPoBS3O1aQd8F25vAWrEsB4RkSItPTObN+esZcTE5azftZ+uTWsy/JxOJLWvh5nF/P1jGQhVgPXh9g6gR4Q2bwH3mNl0YBDwl9wNzGwYMAygWbNmsalURCSO0jKzGDd7Hc9MXMaG3al0b1aT+8/txIntCicIcsQyEPYClcLtqkTonnL34WbWD/gTMMbd90ZoMxIYCZCYmOixK1dEpHClZmQxbvZanpm0nI27U+nRrCb/PK8Lx7etW6hBkCOWgTCHoJtoOtAVWHyQdt8DzYBLYliLiEiRkZqRxX9mBUGwKTmVxOa1ePD8LvRrE58gyBHLQHgXmGJmjYHTgIvNbLi7577j6E/Ao+6eEsNaRETiLjUji9dnruHZr5ezOTmNXi1q88iFXflN6zpxDYIcMQsEd08O7xoaCDzo7puAeRHa3ROrGkREioLUjCzGzljDc18vZ8ueNHq1rM1jF3Wjb6uiEQQ5Yroegrvv5Oc7jURESpX96VmMnbGa5yavYOueNPq0qs0TF3enb+s68S4tomK1QI6ISHGQkp7J2OlreG7yCrbtTaNvqzo8eUl3+rQqmkGQQ4EgIlJAUtIzeWXaap6fsoJte9M5rk0dRgzoQa+WteNdWlQUCCIiR2hfWiavTF/N85NXsH1fOse3rcvvBrQlsUXxCIIcCgQRkcO0Ny2Tl6et4oUpK9kRBsHvT27Lsc2LVxDkUCCIiOTTntQMXp62mhemrGBnSgYntqvH705uS49mteJd2hFRIIiIRGlPagZjvl3FC1NXsislg/7t63HrgLZ0L+ZBkEOBICKSh+TUDF76ZhWjpq5k9/4MTupQn1sHtKVb05rxLq1AKRBERA5i9/4MXvxmJaOnriQ5NZOTjw6CoMtRJSsIcigQRERy2b0/g9FTVzL6m5XsSc1kYMcG/G5AWzo1Kdkz9CsQRERCu1MyGDV1BS9+s4o9aZmc0rEBt5aCIMihQBCRUm9XSjqjpq7kpTAIBh3TkFsGtOGYxqUjCHIoEESk1Nq5L50Xpq5gzLer2ZuWyemdG3LLSW05ulH1eJcWFwoEESl1duxL5/kpK3j521WkZGRxeqdG3DKgDR0als4gyKFAEJFSY/veNJ6fspKXp61if0YWZ3RuxK0D2tKuQbV4l1YkKBBEpMTbtjeN5yev4OVpq0nNzOKsLo255aQ2tFUQ/IICQURKrK170hg5eTmvTl9DWmYWZ3UNgqBNfQVBJAoEESlxtuxJZeTXK3h1xmrSM7P5n25NuPmkNrSuVzXepRVpCgQRKTG2JKfy7NcrGDtjNRlZ2ZzTvQk3929DKwVBVBQIIlLsbU5O5ZlJy3l95hoys51zwjOClnWrxLu0YkWBICLF1qbdqTz79XJem7mGrGxncPcm3NS/DS0UBIdFgSAixc7G3ft5ZtJy3pi5lmx3BvcIgqB5HQXBkVAgiEixsWHXfkZMWsa4WevIduf8Y4/ipv5taFq7crxLKxFiGghmNgroCHzk7sMjPF4LGAvUB+a4+/WxrEdEiqf1u/YzYuIyxs1eiztckNiU3ya1VhAUsJgFgpkNBhLcva+ZjTaztu6+NFezocBYdx9rZq+ZWaK7z45VTSJSvKzbmcLTE5fz1py1wM9BcFQtBUEsxPIMIQkYF25PAPoBuQNhO9DJzGoCTYG1uV/EzIYBwwCaNWsWq1pFpAhZuyOFEZOW8ebsdZQx46KeTbkxqQ1NalaKd2klWiwDoQqwPtzeAfSI0GYqcAZwK/Bj2O4X3H0kMBIgMTHRY1KpiBQJa7an8PTEZbz9XRAEl/Rqxo1JrWmsICgUsQyEvUDO/2JVoEyENvcAN7h7spndBlxFePAXkdJj9fZ9PPXVMsbPXU9CGeOy3s24Iak1jWooCApTLANhDkE30XSgK7A4QptaQGczmw70Br6IYT0iUsSs2raPpyYu450wCIb2ac6NSa1pUL1ivEsrlWIZCO8CU8ysMXAacLGZDXf3uw5o83/Ai0BzYBrwegzrEZEiYuW2fTz51VLe+34DZcsYl/dtzg0nKgjiLWaBEHYDJQEDgQfdfRMwL1ebmcAxsapBRIqWtTtS+PeXS3n7u3WUL1uGK3/TgutPaEV9BUGRENNxCO6+k5/vNBKRUmrT7lSemriU/8xai5lx5W9ackNSK+pXUxAUJRqpLCIxs21vGs9OWs4r01eTle1c1LMpN5/URheLiygFgogUuN0pGYycspwXv1lFakYWg3scxe8GtNXI4iJOgSAiBWZvWiajp67k+Skr2JOayZldGvGHge20ME0xoUAQkSO2Pz2LV6av4plJy9mZksHAjg24bWA7jm5UPd6lST4oEETksKVlZvHGzLU8NXEZW/ekcXzbuvzxlPZ0a1oz3qXJYVAgiEi+ZWRlM/67dfz7y2Ws37WfXi1q89Ql3endqk68S5MjoEAQkahlZTsfzNvA418sYdX2FLo2rck/z+tMvzZ1MbN4lydHSIEgInlydz5btIlHP1/Cks176dCwGs9fnsjJR9dXEJQgCgQROSh3Z9LirTzy+WIWrk+mVb0qPHVpd07v1IgyZRQEJY0CQUQi+nbZNh6esJjv1uyiae1KPHxBV87p1piyCZEmLpaSQIEgIr8wZ/VOHpmwmG+Xb6dh9Yrcf24nLji2KeXLKghKOgWCiACwcP1uHpmwmImLt1K3annuPrMjl/VuRsVyCfEuTQqJAkGklFuyeQ+Pfb6ETxZuokalcvx5UHuu6NuCKhV0eCht9D8uUkqt2raPx79YwnvzNlClfFl+N6At1xzfkuoVy8W7NIkTBYJIKbN+136e/HIpb85ZR7kEY9gJrbj+hNbUrlI+3qVJnCkQREqJLcmpPD1xGa/PXAvA0D7N+W3/1lqTQP5LgSBSwu3Yl86zXy9nzLeryMx2Lkw8iltOakvjmlqTQH7psALBzCq6e6qZlXH37IIuSkSO3O79GYyasoJRU1eSkpHFud2acOuAtrSoWyXepUkRlWcgmNlwIB0wYK27jwZeMbM7gb+b2X3u/mOM6xSRKO1Ly+Slb1fx3NfLSU7N5IzOjfj9yW1p26BavEuTIi6aM4TTgXeAjcCdFkxc0g0oA5RXGIgUDakZWbw6fTXPTFrO9n3pDOhQnz8MbEenJjXiXZoUE9EEwm5gMbA63O4EbAVqAffErjQRiUZ6Zjb/mb2Wp75ayubkNPq1qcttp7SjR7Na8S5NiploAqEhcCKwDdhF0HWUDvQAMs0s2d3XxK5EEYkkMyubd+au54kvl7Ju534Sm9fi8Yu607e11iSQwxNNIDiQFX51BrYDVcLvKwD3AVdGeqKZjQI6Ah+5+/AIj98IXBR+WxOY4e7X5++fIFK6ZGc7Hy7YyOOfL2HFtn10blKD4ed04sR29TQVtRyRaAJhMzCVoMvoCoKAaApUA74AXoz0JDMbDCS4e18zG21mbd196YFt3P0Z4Jmw/ZPAmMP9h4iUdO7OhB8289jnS/hp0x7aNajKs0OO5dRjGigIpEBEEwi1CK4b1APWACuBDIJgGO7uZxzkeUnAuHB7AtAPWBqpoZk1ARq4++wIjw0DhgE0a9YsinJFShZ3Z/LSbTwyYTHz1+2mZd0qPHFxN87s0pgErUkgBSiaQBhPcM2gOvAS8CFBN88KoLaZDXD3LyM8rwqwPtzeQXDN4WBuIjxTyM3dRwIjARITEz2KekVKjBkrtvPIhCXMXLWDJjUr8eD5XRjcvYnWJJCYyDMQ3P2+CH89AsDMJrl76kGeuhfIGQpZleA21V8xszJAf+DOPKsVKSXmrtnJo58vYcrSbdSvVoF//M8xXNizKRXKaipqiZ2oRiqbWX133xJuX+DubwKEo5W7ufv3EZ42h6CbaDrQleDW1UiOJ7iYrE//Uuot2rCbxz5fwhc/bqF2lfLcdcbRDOnTXGsSSKGIduqKV8zsCqAnMMzM3gL6u/tXwOME1wtyexeYYmaNgdOAi81suLvflavdqcDkw6pepIRYtmUPj32xlI/mb6RaxbL87yntuPK4llTVmgRSiKKZuqI5kEZwTaAtwcVkgD8CXxFcYP4Vd082syRgIPCgu28C5kVo99fDqlykBFizPYXHv1zCu3PXU6lcArec1IZr+7WiRmWtSSCF75CBYGb1gDcJ7jDKGY+Au7uZVTCzVgRjESJy9538fKeRiIQ27t7Pk18tY9ystSSUMa49vhXXn9CKOlUP+uskEnOHDAR33wr0MrMJ/PrA3x74O6B7QUWitHVPGiMmLWPsjDW4O5f2bsZN/dvQoLrWJJD4i6bLKGdClFEEoVDDzBKABe4+1Mw+j2WBIiXBrpR0nv16BWO+XUV6Vjbn9ziKWwa04ahaleNdmsh/5dVlVB34FMgGhhCMVD6LYISyZjkVycOe1AxGTV3JqCkr2ZueydldG/O7AW1pVa9qvEsT+ZW8uoySzawvwR1DADsJZjq9GnjNzPoQjF4WkQOkpGcy5tvVPDd5ObtSMhh0TEP+MLAd7RtqTQIpuqIZmJZtZmUJZjudCJzh7uvM7HyCu4wujXGNIsVGakYWr89cw9MTl7NtbxpJ7evxx4Ht6XyU1iSQoi+qm5zd/fRw83sze8jMzN23mNlVwP7YlSdSPGRkZfPm7HU8+dVSNu5OpU+r2jw7pAeJLWrHuzSRqEU7UrkMwUC0L939s5y/d/dZMatMpJiYvGQrd7+3kNXbU+jerCYPX9CV37SuoxlIpdiJdhikE9xi+qWZdQK25ExlIVJa7U7JYPhHP/DmnHW0qleF0Vcm0r99fQWBFFvRdhm5mWWH334ALDCzugS3oT7l7hHXRBApqT5btIm73l3Ijn3p/DapNbcOaKv5hqTYO5yJUta5+9nw39tSJ3GQRXJESppte9O45/1FfDR/I0c3qs6LV/bUIvZSYkQzMK0+wTKYvxLeljqtwKsSKWLcnffnbeDe9xexLy2L/z2lHdef2JpyWpdASpBozhBeJVygJvSLaard/aYCrUikiNm0O5U731nAlz9toVvTmjx0fhfaNtB4Ail5ohmHcAqAmY0ys6lASzN7n2Cw2hh3z4pxjSJx4e68MWstD3z0IxnZ2dx1xtFcdVxLLVspJVZ+riHMd/fjAcysEXA98I2Zna07jqSkWbsjhTvGz+ebZdvp26oO/zyvM83rVIl3WSIxFe04hAQOWALT3TcC95rZZ8CbZtbf3bMP+gIixURWtjPm21U89NliEsoYD5zbmYt7NqWMzgqkFMjPOIRfra3s7tPMbKjCQEqCZVv2cvvb85mzeif929fj/nM707hmpbyfKFJCRBsICcDKcPW0fcA+d98P4O6a3E6KtYysbEZOXsETXy6lcvkEHr2wK+d2b6IBZlLqRBsIrYC3CSa3qwhUNLPKQB1glbtfGZvyRGJr0Ybd/Pmt+SzakMzpnRvy97M7Ua+aVi2T0ik/F5VnufstAGbWGVgUzoS6xMzquPv22JQoUvDSMrN48stlPPv1cmpWLs+zQ3owqFOjeJclElfRDEx7i6CbKMHMqgJjgUzgJmATMDz8XqRY+G7NTm5/az5Lt+xlcI8m/O3MjtSsXD7eZYnEXTRnCJcAA4EbgUeA37v7ypwH3f3lGNUmUqD2p2fx8ITFjP5mJY2qV+TFq3rSv339eJclUmREMzAtA/g4XB2tPTA018W2FHd/ONJzzWwUwbQXH7n78IO9h5mNAD5x9w/yU7xItL5dvo073l7Amh0pDOnTjNsHdaBaxXLxLkukSMlzIpbwziKAM4ERwLkEE9pNBs4BphzkeYOBBHfvC7Qys7YHaXc80FBhILGwJzWDv76zgEufn0EZgzeG9WH4OZ0VBiIRRNNldJ2ZHQvscvevzWynu08GMLNd7j7jIM9LAsaF2xOAfsDSAxuYWTngeYIzkP9x9/dyv4iZDQOGATRr1iyKckUCE3/awl/fWcDm5FSuO74ltw1sT6XymqJa5GCi6TK6C8DMdpjZ50BnM5sAGNDFzD5190ERnloFWB9u7wB6RGhzOfAD8CBwi5k1c/cnc73/SMLJ9RITE/3XLyHySzv3pXPfhz/wztz1tGtQlWeGHEe3pjXjXZZIkZef207rRhqRHC6vGcleIGeYZ1Uid091B0a6+yYzexW4H3gyQjuRqHy8YCN/e28hu1IyuPWkNtx0UhsqlNVZgUg0og6EcMzBScBkd880s/Pd/a1DTFsxh6CbaDrQFVgcoc0ygkFvAInA6uhLF/nZlj2p/O3dRXy6aBOdmlTn5at707Fx9XiXJVKsRDMO4R3gX+4+neC207vNrALBbKdvHeKp7wJTzKwxcBpwsZkNz+mCCo0CRpvZxUA54PzD/HdIKeXujP9uPfd9+AP7M7K4fVAHrju+JWW1cI1IvkVzhtAEuNTM7gLeIfi0nwBUM7MTgXLu/kXuJ4WrqSURjGF40N03AfNytdkDXHBk/wQprdbv2s9fxy/g6yVbSWxei3+d34XW9arGuyyRYiuaQNgFfAA8Hf7ZK3xedaAnwbWBXwUCgLvv5Oc7jUQKRHa2M3bmGv758Y84cO9ZHbm8bwtNUS1yhA4ZCOE6CN2BvuGfnxIEQzbQ6GAD0kRiZdW2fdz+9nxmrNxBvzZ1+b/BnWlau3K8yxIpEfI6Q6gFXEXQRdSU4NP+x8CpwEsxrUzkAFnZzuipK3nk88WUSyjDv87rzIWJTTVFtUgByisQTgVaE4xK7gWcR3BnEO7+GoCZlXf39FgWKaXbks17+NNb85m3dhcnH92A+8/tRIPqFeNdlkiJk1cgTAOeIbglFKAzwXWEKWY2lGBwWgJweswqlFIrPTObZyYt56mJS6lWsRz/vqQ7Z3VppLMCkRjJKxDWESyf+ThwA7AAuAe4l2DCuudiWp2UWgvW7eZPb83jp017OKtrY+49qyN1qmrhGpFYOmQguHu6mV0JfAdcBgwM5zMaBNxmZlXdfW8h1CmlRGpGFo9/sZTnp6ygTpXyPH95IgM7Noh3WSKlQjS3ne4nuI7wk7t/bGZ9gVlAlsJACtKsVTu4/a35rNi2jwsTj+LOMzpSo5JmJRUpLNEM53wSOAv4T/j9/xF0IyXFqCYpZfalZXLPewu58LlppGVm88o1vXjw/K4KA5FCFs0ZwgZ3v9fM+oUL3nQAXiWY6fR1oLq7nxHTKqXEmrJ0K38Zv4D1u/ZzRd8W/OnU9lSpkJ85F0WkoETzm9fRzN4H3iSYuuIU4GpgnLtfEsvipOTavT+D+z/6gXGz19GqbhXGXd+Xni1qx7sskVItr5HKZYD33P3acKGar4G7gFuANYVQn5RAn/+wmTvfWcD2fenccGJrfn9yWyqW0xTVIvGW111G2Wb2r3AZzcuBmwkGpnUHZpnZeGCpu98e+1KluNu+N417P/iBD+ZtoEPDaoy6oiedj6oR77JEJBRNl1Fb4AGgMbAQWB7+3VIgk2Caa5GDcnc+mL+Re99fxJ7UDG4b2I4bTmxN+bKaolqkKMmry+gS4FaCO4tuAVYAEwlGLL8JpBAEhEhEm5NTufOdhXzx42a6HlWDB8/vQ/uG1eJdlohEkFeX0etmtha4L2ybQLB+wS6C0coALwPjY1ijFEPuzrjZaxn+0Y+kZ2bz19M7cPVxWrhGpCjLs8vI3aea2ZnAte7+7wMfM7OKQLtYFSfF09odKfxl/AKmLttGr5a1+dd5XWhZt0q8yxKRPER1w7e7pwD/NrP+7j7xgL9PBebHqjgpXrKznZenreLBzxZjwD/O6cRlvZpp4RqRYiKaNZWfBYa7+zrgDjPLfc2ggrsvjUl1Umws37qX29+az+zVOzmxXT0eGNyZJjUrxbssEcmHaM4QlhKsq7yO4BrC3bker0BwS6qUQplZ2YycsoLHv1hKpXIJPHxBV87r0URTVIsUQ9EGQqNwO8Xdr4thPVKM/LAhmT+/PY+F65MZdExD7jvnGOpX08I1IsVVNIGwAnjMzK4DEszsUyAd2Am86+7vxLJAKXrSMrN4+qtljJi0nJqVyzHish6c3rlR3k8UkSItmkBoApwB1AS6AhkEK6UtAG40sxuBs8MLzFLCzV2zkz+/NZ+lW/Zybvcm/O3MjtSqUj7eZYlIAYgmEAYQTFfRCRgEfA/c4u6dgH+YWctIYRDOjNqRYGW14REeL0tw9rEi/Ktb3H3B4f0zJNb2p2fxyITFjP5mJQ2qV+TFK3vSv0P9eJclIgUomkCYBRwPVAIWAZOAq8ysrLtnuvvK3E8ws8FAgrv3NbPRZtY2wp1IXYDXNQ9S0Td9xXZuf3s+q7encGnvZvzltA5Uq6i1CkRKmmgCYQ5wJ5ANZBF0G60APjWzBOBNdx+R6zlJwLhwewLQj+Di9IH6AGeaWX+C7qfr3T0z95uHs6wOA2jWrFkU5UpB2ZOawT8/+YmxM9bQrHZlXruuN79pXTfeZYlIjEQzUnkFcE2kx8ysAtA3wkNVgPXh9g6gR4Q2s4CT3X2jmb0MnA68H+H9RwIjARITEz2veqVgTFq8hb+OX8DG5FSu6deSP57SjsrltXCNSEmWr99wM7vG3UflfO/uaQRdSLntJehiAqhK5KU654fPB5hNMIOqxJm78+RXy3j08yW0qV+Vt274Dcc2rxXvskSkEOR3prELomw3h6CbCIIuplUR2rxiZl3DbqdzgHn5rEUKWHa2848Pf+TRz5cwuHsTPrq1n8JApBTJbx9AdpTt3gWmmFlj4DTgYjMb7u53HdDmPuA1gltY33f3L/JZixSgzKxs7hi/gLfmrOPK37Tgb2d21BxEIqVMNHMZfQOkERy4O5vZV+G2h39WdPdfXEdw92QzSwIGAg+6+yZynQG4+0KCO40kzlIzsvjdG3P5bNFm/nByO24d0EZTT4iUQtFcVD4uZ9vMPnH306J5YXffyc93GkkRtTctk+tfmc03y7Zzz1kdueq4lvEuSUTiJL9dRv9dCd3Mmrv76gKuRwrRzn3pXPnSLBau380jF3TlvGOPindJIhJHUV1UNrM24eYHB/z1g+H8RlIMbU5O5aKR0/hxYzLPXNZDYSAiUV1D6Ac8Eq6adnY4CtkJzhaeMbMUdx8b4zqlAK3evo/LXpjBzn3pvHRVTw02ExEgui6j3sBp7r7DzLLdvX/OA2ZWlejvPJIi4MeNyVw+eiaZWdm8dl0fujatGe+SRKSIiOai8iMHfPsxBGsphxPapbi7AqGYmLN6J1e9OJPK5cvy2vV9adugWrxLEpEiJM9rCGY23Mz+Zmb3EIxAhmBQWTtgrJkdHdMKpUBMWbqVIS/MoHaV8rx5g8JARH4tmi6j04F3gI3AX8P707sRhEl5d/8xduVJQfhkwUZufWMuretV5eVremlVMxGJKJpA2A0sBlaH252ArUAt4J7YlSYFYdystdwxfj7dm9Vi9JU9qVFJ01aLSGTRBEJD4ERgG7CLYHRyOsEMpplmluzua2JXohyu5yev4P6Pf+SEdvV4dkgPzVYqIocUzTgEJ1gHIQvoTLCkZpXw+woEcxJJEeLuPPTZT9z/8Y+c0bkRL1yeqDAQkTxFEwibganAZ0AyQUA0BaoBU4BbYlad5Ft2tnP3ewt5euJyLunVlH9f0p3yZfM7qa2IlEbRfGysRXDdoB6wBlgJZBAEw3B3PyN25Ul+ZGRl88dx83h/3gauP7EVdwzqoEnqRCRq0QTCeIJrBtWBl4APgYsIltGsbWYD3P3LmFUoUdmfnsVNr33HVz9t4fZBHbgxqXW8SxKRYiaagWmRrhGMADCzSeEANYmj5NQMrn1pNrNW7+CBcztzaW+tPS0i+XdEVxoVBvG3bW8aV4yeyZLNe/j3xd05q2vjeJckIsWUbj0pxtbv2s/QF2awYfd+nr88kaT29eNdkogUYwqEYmr51r0MfWEGe9IyefWa3iS2qB3vkkSkmFMgFEML1+/m8tEzKWPwxrA+HNO4RrxLEpESQIFQzMxYsZ1rxsymRqVyvHJNL1rVqxrvkkSkhFAgFCNf/bSZG1/9jqNqVeLVa3vTqEaleJckIiWIAqGYeO/79fxx3DyOblSdMVf3onaV8vEuSURKmJjOaWBmo8xsmpndlUe7BmY2N5a1FGevTF/N7//zPcc2r8Vr1/VWGIhITMQsEMK1lxPcvS/QyszaHqL5w4D6P3Jxd576ail3v7uQAR3qM+bqXlSrqOmrRSQ2YnmGkASMC7cnAP0iNTKzk4B9wKYY1lLsuDsPfPwjD09Ywrndm/DMkGOpWC4h3mWJSAkWy0CoAqwPt3cADXI3MLPywN3AHQd7ETMbZmazzWz21q1bY1JoUZOZlc3tb8/n+SkrufI3LXjkgq6US9CMpSISW7E8yuzl526gqgd5rzuAEe6+62Av4u4j3T3R3RPr1asXgzKLlrTMLG5+bS7jZq/jdwPacs9ZHSlTRjOWikjsxTIQ5vBzN1FXYFWENicDN5nZJKCbmb0Qw3qKvH1pmVzz0mw+XbSJv53ZkT8MbKfpq0Wk0MQK7tjLAAAPWUlEQVTyttN3gSlm1hg4DbjYzIa7+3/vOHL3E3K2w5lTr41hPUXarpR0rnppFvPW7uKh87twQWLTeJckIqVMzALB3ZPNLAkYCDzo7puAeYdonxSrWoq6LcmpDB01k5Xb9vHMkGM59ZiG8S5JREqhmA5Mc/ed/HynkUSwZnsKQ0bNYNveNF68qifHtakb75JEpJTSSOU4WrxpD0NHzSA9K5vXrutDt6Y1412SiJRiCoQ4mbtmJ1e+OIuK5cow7vq+tGtQLd4liUgpp0CIg6lLtzHsldnUq1aBV6/pTdPaleNdkoiIAqGwfbpwI7e+/j2t6lXh5at7Ub96xXiXJCICKBAK1bjZa7nj7fl0a1qTF6/sRY3KmpdIRIoOBUIheWHKCoZ/9CPHt63Lc0OPpXJ57XoRKVp0VIoxd+exz5fw76+WcXrnhjx2UTcqlNUkdSJS9CgQYig72/n7B4sYM201FyU25YHBnUnQvEQiUkQpEGIkIyubP705j3e/38CwE1rxl9M6aF4iESnSFAgxkJqRxU1jv+PLn7bwp1Pb89uk1goDESnyFAgFbE9qBteOmc3MVTsYfk4nhvRpHu+SRESiokAoQNv3pnHFizP5aeMenri4O2d3bRzvkkREoqZAKCAbdu1nyKgZrN+5n+cvT6R/h/rxLklEJF8UCAVgxda9DB01k+T9GbxyTW96tawd75JERPJNgXCEFm3YzRWjZ+IOrw/rQ6cmNeJdkojIYVEgHIFZq3Zw9YuzqFaxLK9e25tW9arGuyQRkcOmQDhME3/awo1j59C4ZiVevaY3jWtWindJIiJHRIFwGN6ft4Hb/vM9HRpVY8xVvahTtUK8SxIROWIKhHx6dfpq7n5vIT1b1GbUFYlUq6gZS0WkZFAgRMndGTFpOQ99tpgBHerz9GU9qFhOk9SJSMmhQIiCu/PPT37iuckrOKdbYx66oCvlEsrEuywRkQIV90Aws9rAscBcd98W73pyy8p27nxnAW/MWsvlfZtz71nHUEYzlopICRTTj7lmNsrMppnZXQd5vBbwIdALmGhm9WJZT36lZWZx6+tzeWPWWm45qQ1/P1thICIlV8wCwcwGAwnu3hdoZWZtIzTrAtzm7vcDnwE9YlVPfqWkZ3LtmNl8tGAjd51xNH88pb1mLBWREi2WZwhJwLhwewLQL3cDd//a3aeb2QkEZwnTcrcxs2FmNtvMZm/dujWG5f5sd0oGQ16YwTfLtvHgeV249vhWhfK+IiLxFMtAqAKsD7d3AA0iNbLgY/dFwE4gI/fj7j7S3RPdPbFevdj3KG3Zk8pFI6excH0yIy7rwYU9m8b8PUVEioJYBsJeIGf4btWDvZcHbgLmA2fHsJ48rd2RwgXPTmPNjhRGX9mTQZ0axbMcEZFCFctAmMPP3URdgVW5G5jZ7WZ2efhtTWBXDOs5pCWb93D+s9+yKyWDsdf2pl/buvEqRUQkLmIZCO8CQ83sUeBCYJGZDc/VZmTYZjKQQHCtodDNW7uLC5+bhjuMu74v3ZvVikcZIiJxFbNxCO6ebGZJwEDgQXffBMzL1WZn+HjcfLt8G9eNmU2dqhV49ZreNKtTOZ7liIjETUwHpoUH/HF5NoyTCYs2cfPrc2lZpwqvXNOL+tUrxrskEZG4iftI5Xh5e846/vz2fDo3qcFLV/WkZuXy8S5JRCSuSmUgjJ66kvs+/IF+bery3NBjqVKhVO4GEZFfKFVHQnfn8S+W8sSXSxl0TEOeuKQbFcpqxlIREShFgZCd7dz34Q+89O0qLjj2KP5vcGfKasZSEZH/KhWBkJGVze1vzWf83PVc268ld55xtOYlEhHJpVQEwphvVzF+7nr+95R23NS/jcJARCSCUhEIl/dtQfM6VRjYMeJ0SiIiQozXQygqypctozAQEclDqQgEERHJmwJBREQABYKIiIQUCCIiAigQREQkpEAQERFAgSAiIiFz93jXEDUz2wqsPsyn1wW2FWA5BUV15V9RrU115Y/qyp8jqau5u9fLq1GxCoQjYWaz3T0x3nXkprryr6jWprryR3XlT2HUpS4jEREBFAgiIhIqTYEwMt4FHITqyr+iWpvqyh/VlT8xr6vUXEMQEZFDK01nCCIicggKBJESwMxqm9lAM6sb71qk+FIgFCIza2BmUw7xeBMzW2dmk8KvPO8bLonMrIaZfWJmE8zsHTMrH6FNWTNbc8C+6lxItRW5A6+Z1QI+BHoBEyP93MRrfxVl4e/j3IM8Vir3V4kMhCgOvOXM7AMz+8bMri6kmmoBY4Aqh2jWG7jf3ZPCr60xrinPA2/YbpSZTTOzu2JZzwEuAx5191OATcCgCG26AK8fsK8WxLqoaA68YbvC3l9dgNvc/X7gM6DHQdoU6v7KcagDb/h4Ye+vHA8DlQ7yWDx+vqIKITP7u5nNMrOnC7qGEhcIUR54bwHmuPtxwPlmVq0QSssCLgKSD9GmD3CtmX1nZg8UQk15HnjNbDCQ4O59gVZm1jbWRbn7CHf/PPy2HrAlQrM+wJlmNjM8oBTGcrB5HnjjtL++dvfpZnYCQVhNi9AsHvsrx0EPvPHYX+H7ngTsI/i5jyReP1+HDCEzOxboR/D/vMXMTi7IAkpcIBDdgTcJGBduTwZiPirR3ZPdfXcezT4hqK0n0NfMusS4pmgOvEn8vK8mEPwwFgoz6wvUcvfpER6eBZzs7r2AcsDpsa4nygNvEnHYX2ZmBD/3O4GMCE0KfX+FdeV14E2ikPdXeCZ8N3DHIZrFY39FE0InAm97cHvoZ8DxBVlAiQuEKA+8VYD14fYOoKgsuPytu+9x9yxgLlBYn5YOdeCNy74ys9rAk8DBuvTmu/vGcHs2hbev8jrwxmV/eeAmYD5wdoQmhb6/ojzwxmN/3QGMcPddh2gTj5+vaEIopvurxAVClPby8ylsVYrOfvjMzBqZWWXgFGBhrN8wigNvoe+r8EDyJvAXdz/YZIavmFlXM0sAzgHmxbouiOrAG4/9dbuZXR5+WxOIdKCLx/6K5sAbj9/Fk4GbzGwS0M3MXojQJh77K5oQiun+KioHwsI2h59PTbsCqwq7ADM7ycxuzvXXfwcmAtOBZ919cYxriObAG499dQ1B//yd4cW1e8xseK429wGvAN8D09z9i1gXFeWBNx77ayQw1MwmAwnAuqKwv4juwFvo+8vdT8jppyfYH48Wkf0VTQjFdH+V2JHKZjbJ3ZPCPsyO7v7UAY81Bz4GvgB+A/QJu2lKFTO7EXiAn3/wJgLl3P2uA9pUB6YAXwKnEeyrvLrkSqTwhoVxQAWCs7engUu0v/IWhsJvgUu1vyIzs07Aa4AB7wOPAA+6+7UHtClDsL9mE9wEMsjdVxZYDSU1EPJiZo0Jkvaz0voDGK3wQDgQmOzuB7s4KCHtr/zR/sofM6sEnAF85+4rCvS1S2sgiIjIL5XWawgiIpKLAkFERAAFgpQwZlYmHCsQ8fsCeP2EgnotkaJGgSDFipmdYmZHhdtPm1lzM/vggCYDgfVmNt3MvgE2AA+Y2WNm9nD4VfaA17vfzFqG2+XN7O1DvHdV4GszqxllrU+YWQ8zGxPpOWZ2pZlVNbNBZnZqhMfvM7P+YY13mFk1M/ssdyiZ2RAzW2JmX+T6WmFml0RTqwgoEKT4qQDcYmaJwLnAC0BieKC82d0/A8YDFwP/A3xKcAtfAvA4wW2NmQe83rFAzhiMgUCKmXUIv8oDmFlFMzN33ws8Bpyf82Q7yISAoYrhny8BV+Q6c2lOMKfWPoJR6cPNrOIBj1clmH6lL1CfYJBSc2Cfu2eFZz45v7/pwEh3P/nAL+BlgqlcRKJSmBNciRSED4G1QHuCof2bgKfc/fxc7d4gOBguBdYBFd19nZmlwn+7frYRjMH4LjzL6E5wEL6DYK6iwcBPBKGSZmZOcFDONLOc9ytvZgPDg/RLQGuCgzzhdjeCAWwVgP/w85w+TwJ/Deek2WxmrwJjzexid88AagB1gJsJBkd9G263CQegtSEYvDQTyAaGmVnuyQlbAX+Oes9KqadAkGLDzE4DbgO+BtIIPql3BLLM7EuCg/nvw+YXEwzzfxjYCDQKP5VvAAgP4N+5+wAzSyIYNLUR+KO7J1swtXBa2DbpgBp+D+xy95cilJgNXOfuP4VtbyIIo4VA15x77M3sDwQhMSHnie7+hJk1Ab4xs2sJJhpsQTA4qT3QkGA2zDuBZcD17j7zgPce6e4P59pf9x56j4r8kgJBig13/8TMfgT+7u45XTBfEUx3Ts5BOuyZeZXgDGGlu2eaWTngQg44CANdwxG0NYF3CaYqeJVgjqIKhJ/0zewyYBjBZHZNCc4QhgDlgSfd/c2cEgk+5RvwHcHZTCuCs41l4Ws1B04ClgALLVhoZyVBlxbAUwQTmJUlmMpkKPBo+N4PEHRxpQMFOiBJBBQIUjzljKYcRDC3CwQTkZ5JcKAFGEJ4hmDBBH6fAn8Ajj7gdb5395PDM4Qkd19uZlkWTDteiTAQ3H0sMDZ8k0OdIVQi6GZqEP45j+DMozzwt/C1VgNnha81B2jj7veFQfGou48JHzsOGE5w7aArQddTS4LrIxBMlX4gdRnJEVMgSHGUYGZdgYeA/gRdKi8SfLJOIpgF8jKCA+IJwKlAXYIzhgYEn8gP5iZgM8HBPSWfdTUDtgPtCKYmXkVwMB96kPbnA/8KtxsTXBsBwN2/MbNxBHPkfwIscvcMM/uO4NrBfbleS11GcsQUCFLc3EewHsECgonltgJbCacKNrPTCQ7mswnusikLVCb4hH0G8I6Z3RCu/dA9V5cR7r4hfJ067u7hnT9l3D1iOBxwC2tloDrB/FgPAvcQ3AE1heCMoL6ZVXf3EeHzbgUy3X1G+Px2HBAIoWcJzlLuAlaHt8ceQ3Bto0f4b4RgMrSDnSHcfvBdKfJLCgQpNixYY7Y2wcH/MoLuoDIEt3dWJLjwere7XxS2b0jQXVONYJbNZDO7InjIEgiWUT3FggWCjg+fcwXBAThnsaBTCW5zzV3LkHAzgeDspBFBAH0DXE8wE+oj7j7WzMYCTwCXhdcXxhOcSQwJX+seglteLz/g9WsBzxBcK+hDcPH8ReB/Cc5g3jKzS919OcHv8Qh3fzxXjfei33HJB01uJ3KA8EBcNjzzyM/zygGE3ToGNDxgsRPMrJ27Lwm3q7j7voO81IGvWTZnzET4mmVypmkPx0Xol1cKlAJBREQAjVQWEZGQAkFERAAFgoiIhBQIIiICwP8D69Ni8dubMSoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 可视化 所解释的方差与选取的主成分数目之间的关系\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['font.sans-serif']=['SimHei']\n",
    "%matplotlib inline\n",
    "\n",
    "\n",
    "plt.plot(np.array(range(len(compressor.explained_variance_ratio_))) + 1, \n",
    "         np.cumsum(compressor.explained_variance_ratio_))\n",
    "plt.xlabel('选取的主成分数目')\n",
    "plt.ylabel('累计所解释的方差累')\n",
    "plt.show(); # 前5个主成分可以保证保留原特征中90%的方差"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Supervised Methods 监督方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LDA (Linear Discriminant Analysis) 线性判别分析"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与主成分分析（PCA）不同的是，线性判别分析（LDA）是一种有监督机器学习模型，旨在找到特征子集以最大化类线性可分离性，即希望投影望同一种类别数据的投影点尽可能的接近，而不同类别的数据的类别中心之间的距离尽可能的大。线性判别分析仅适用于分类问题，其假设各个类别的样本数据符合高斯分布，并且具有相同的协方差矩阵。\n",
    "\n",
    "可以在sklearn的[官方网站](https://scikit-learn.org/stable/modules/lda_qda.html#lda-qda)上了解更多原理方面的详细信息。LDA会将原始变量压缩为（K-1）个，其中K是目标变量类别数。但是在sklearn中，通过将主成分分析的思想合并到LDA中，其可以进一步压缩变量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-03-30T03:37:21.167912Z",
     "start_time": "2020-03-30T03:37:21.148967Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA\n",
    "\n",
    "# LDA仅适用于分类问题\n",
    "# 载入数据集\n",
    "from sklearn.datasets import load_iris\n",
    "iris = load_iris()\n",
    "X, y = iris.data, iris.target\n",
    "\n",
    "# iris 数据集使用前需要被打乱顺序\n",
    "np.random.seed(1234)\n",
    "idx = np.random.permutation(len(X))\n",
    "X = X[idx]\n",
    "y = y[idx]\n",
    "\n",
    "# 选择前100个观测点作为训练集\n",
    "# 剩下的50个观测点测试集\n",
    "\n",
    "train_set = X[0:100,:]\n",
    "test_set = X[100:,]\n",
    "train_y = y[0:100]\n",
    "test_y = y[100:,]\n",
    "\n",
    "# 在使用主成分分析前，我们需要先对变量进行缩放操作\n",
    "# 因为LDA假定数据服从正态分布\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler # 我们也可以采用幂次变换\n",
    "model = StandardScaler()\n",
    "model.fit(train_set) \n",
    "standardized_train = model.transform(train_set)\n",
    "standardized_test = model.transform(test_set)\n",
    "\n",
    "# 开始压缩特征\n",
    "compressor = LDA(n_components=2) # 将n_components设置为2\n",
    "# n_components <= min(n_classes - 1, n_features)\n",
    "\n",
    "compressor.fit(standardized_train, train_y)  # 在训练集上训练\n",
    "transformed_trainset = compressor.transform(standardized_train) # 转换训练集 (20000,2)\n",
    "transformed_testset = compressor.transform(standardized_test) # 转换测试集\n",
    "assert transformed_trainset.shape[1] == transformed_testset.shape[1]\n",
    "# 转换后训练集和测试集有相同的特征数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-03-30T03:37:21.314293Z",
     "start_time": "2020-03-30T03:37:21.169760Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEHCAYAAAC5u6FsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd4VHXaxvHvQ+hVeu+9BBADiKICNhAblrXXVdRV13eLgIKKigXXtru2xVVXV9ddpSpNRUVQQQUWE3rvvZcQ0p73jzOsERMygUwmydyf68rFyZkzmeeEZO6c3zm/55i7IyIiEq4S0S5ARESKFgWHiIjkiYJDRETyRMEhIiJ5ouAQEZE8UXCIiEieKDhERCRPFBwiIpInCg4REcmTktEuIBJq1KjhTZo0iXYZIiJFyty5c3e4e83ctiuWwdGkSRPmzJkT7TJERIoUM1sbznYaqhIRkTxRcIiISJ4oOEREJE8UHCIikicKDhERyRMFh4iI5ElEg8PMapvZzGM8XsrMPjazb8zs1rysExGRn8vMLJg7ukYsOMysKvA2UOEYm90LzHX304ErzKxSHtaJiEjI3LW76ffnmcxbtzvirxXJI44M4Cpg3zG26QV8EFqeASTkYd3PmNlAM5tjZnO2b99+gqWLiBQNBw+n8+jHC7nitW85cDid1PTMiL9mxGaOu/s+ADM71mYVgI2h5V1A7TysO/r1RgGjABISEgrmeE1EJIpmLt/OA2OT2LD7EDf2aMygvm2oWCbyDUGi3XLkAFAO2AtUDH0e7joRkZi0NzmNJyYv4oM5G2hWowIf3NGDbk2rFdjrR/uqqrlAz9ByJ2BNHtaJiMScqQu2cM4LXzFm3kZ+06s5k+87o0BDAwrwiMPM+gDt3P2lLKvfBiab2RlAO+A7giGpcNaJiMSMbftTGP7RQiYnbaFd3cq8dXNXOtSvEpVazD26pwPMrB7B0cQn7r43L+tykpCQ4OqOKyLFgbszdt5GHpu4iENpGdx3dksGntmMUnH5P2BkZnPd/RcXHx0t2uc4cPdN/HTFVJ7WiYgUZxt2J/PguAXMWLadUxpXZeTlHWlRq2K0y4p+cIiIyM9lZjrvfreWkVOW4MCjF7fnhlMbU6LEMa9SLTAKDhGRQmTl9gMMGZPID2t2c2armjw5oAMNqpaPdlk/o+AQESkE0jIyGTVjFX/+fDnlSsXx7JWduLxL/dzmwkWFgkNEJMoWbNzL4DGJLNy0jwvi6zD84vbUqlQ22mXlSMEhIhIlKWkZ/OXz5fxtxiqqli/Na9d3oW+HutEuK1cKDhGRKJizZheDxiSyavtBrjylAcP6t6NK+VLRLissCg4RkQJ04HA6f5q6hHdmr6X+SeV459ZunNmqZrTLyhMFh4hIAflq2XYeHJvEpr2HuKlHE+4/vzUVCqApYX4rehWLiBQxe5JTeWziIsbO20jzmhUYfWcPTmlcsP2l8pOCQ0QkgiYnbebhCQvYk5zGPb1bcE+fFpQtFRftsk6IgkNEJAK27Uvh4QkLmbpwCx3qV+btW7vRvl50mhLmNwWHiEg+cnc+nLuBERMXkZKeyeC+bbj9jKaUjEBTwmhRcIiI5JP1u5J5cFwSM5fvoFuTajx9eTzNaka/KWF+U3CIiJygjEznnVlreGbqUkoYPH5pB67r1qjQNCXMbwoOEZETsGLbfgaNTmTeuj30al2TJwbEU/+kctEuK6IUHCIixyEtI5O/fbWSv3y+gvJl4njhqk5c2rlwNiXMbwoOEZE8Stqwl/tH/8iSLfvp37Euj17cnhoVy0S7rAKj4BARCVNKWgYvTlvO6zNXUb1Caf52wymc375OtMsqcAoOEZEwfLdqJ0PGJrF6x0Gu7tqQBy5oS5VyRaMpYX5TcIiIHMP+lDRGTl3Cu7PX0bBaOd67rTunt6gR7bKiSsEhIpKDL5dsY+i4JDbvS+HXPZvyh/NaUb603jb1HRAROcqug6k8PnER4/67kZa1KjLmrtPo0qhqtMsqNBQcIiIh7s6kpM08MmEhew+l8duzW3J37+aUKVm0mxLmNwWHiAiwdV8Kw8Yv4LNFW+nYoArv3tadtnUrR7usQknBISIxzd35zw/reWLyYlLTMxl6QVtuOb1JsWpKmN8UHCISs9btTGbI2ES+XbmT7k2rMfLyjjSpUSHaZRV6Cg4RiTkZmc5b36zm2U+XUrJECZ4cEM/VXRsW26aE+U3BISIxZdnWoCnh/PV76NOmFk8M6EDdKsW7KWF+U3CISExITc/k1ekreenL5VQqW4o/X92ZizvVi4mmhPlNwSEixd6P6/cwaHQiS7fu55LO9Xj4wnZUj6GmhPlNwSEixdah1Aye/2wpb3y9mlqVyvL3GxM4p13taJdV5Ck4RKRYmrVyJ0PGJrJ2ZzLXdm/EkH5tqFw2NpsS5jcFh4gUK/tS0nhq8hLe/34djauX51+3d+e05rHdlDC/RSw4zOwNoB0wyd1HZPN4U+AloDLwvbv/IYd1VYH3gFrAXHe/I1I1i0jR9vnirQwdt4Bt+1MYeGYzfndOK8qVVruQ/BaRqZFmdhkQ5+49gGZm1jKbzUYCj7v7GUADM+uVw7obgPfcPQGoZGYJkahZRIqunQcO89v3/8uv355DlXKlGPub03nwgrYKjQiJ1BFHL+CD0PKnQE9g+VHbtALmhZa3AVVyWLcT6GBmJwENgfXZvaCZDQQGAjRq1Cg/9kFECjl356MfN/Hox4vYn5LG785pxV29mlO6pNqFRFKkgqMCsDG0vAvoks02o4FHzGw20Bd4AIjPZl11oD/wW2Bx6Ov9gruPAkYBJCQkeL7tiYgUSpv3HmLYuAV8vmQbnRuexDNXdKRV7UrRLismRCo4DgBHpmJWJJshMXcfYWY9gfuBt939APCLdWb2F+BOd99nZr8HbiEUECISezIznfd/WMdTk5eQnpnJsP5tueX0psSpXUiBiVRwzCUYnpoNdAKW5rDdfKARcM0x1lUF4kNHId2BaZEoWEQKvzU7DjJkbCKzV+3itObVefqyjjSqXj7aZcWcSAXHeGCmmdUD+gFXm9kIdx921Hb3A8+7e/Ix1j0FvAU0BmYB70eoZhEppNIzMnnzm9U89+kySseV4OnL4rmqa0O1C4kSc4/M6YDQZbTnAjPcfUtEXiQHCQkJPmfOnIJ8SRGJkCVb9jF4dCI/btjLOW1rM+LSDtSpUjbaZRVLZjY3dAXrMUVsHoe77+anK6tERPLkcHoGL3+5kle+XEGVcqV46dqT6R9fV0cZhYBmjotIoTNv3W4Gj05k+bYDDDi5Pg9f2I6qFUpHuywJUXCISKGRnJrOc58u481vVlOnclneurkrvdvUinZZchQFh4gUCt+s2MGQsYms33WI609txOC+baikpoSFkoJDRKJq76E0npq8mH//sJ6mNSrwn4Gn0r1Z9WiXJceg4BCRqPl04RaGjV/AzoOp3HlWc/7vnJaULaX+UoWdgkNECtz2/YcZ/vFCJiVupm3dyrxxU1fiG1SJdlkSJgWHiBQYd2f8/I08+vEikg9n8MfzWnHHWc0pFaemhEXJcQWHmZV19xQzK+HumfldlIgUPxv3HGLouCSmL91Ol0ZBU8IWtdSUsCjKNTjMbASQChiw3t3fBP5pZkOBR83sMXdfHOE6RaSIysx03vt+HU9PXkymwyMXtePGHk3UlLAIC+eI4wJgHLAZGGrBtM3OBB1vSys0RCQnq7YfYMiYJL5fs4ueLWrw1GXxNKympoRFXTjBsZegu+3a0HIHYDtB19pHIleaiBRV6RmZvD5zNS9MW0bZkiV45oqOXHlKA7ULKSbCCY46wFnADmAPwZBVKsHNmdLNbJ+7r4tciSJSlCzatI9BY35kwcZ9nN++No9f0oFaldWUsDgJJzgcyAh9xBPcyrVC6PMywGPAzRGqT0SKiJS0DF76YgWvfbWSk8qX5tXrutAvvm60y5IICCc4tgJfEwxV3UQQJA2BSgQ3VXorYtWJSJEwd+0uBo1OZOX2g1zepQEPXdiWk8qrKWFxFU5wVCU4r1ETWAesBtIIAmSEu/ePXHkiUpgdPJzOnz5Zytuz1lCvSjnevrUbZ7WqGe2yJMLCCY6xBOc0KgP/ACYCVwGrgGpmdra7fx6xCkWkUJqxbDsPjE1i455D3NSjMff3bUPFMppTHAty/V9298eyWf0KgJlNd/eUfK9KRAqtvclpPD5pEaPnbqBZzQp8eGcPujapFu2ypACF9eeBmdVy922h5Svd/UOA0Ozxzu4+P5JFikjhMHXBZh6asJBdB1P5Ta/m/PZsNSWMReE2iPmnmdUxs4uAgRboE3rsxQjVJiKFxLb9Kdz17lzufHceNSuWYcLdpzOobxuFRowKp+VIY+AwwSW4LQlOigP8AfiC4ES5iBRD7s6YeRt5fOIiDqVlcP/5rRl4ZjM1JYxxxwwOM6sJfEhwRdWR+Ry4u5tZGTNrRjCXQ0SKmQ27k3lw3AJmLNtOQuOqPH15R1rUqhjtsqQQOGZwuPt2oJuZfcovA6I18CjQKEK1iUgUZGY6/5y9lpFTl2DAY5e05/rujSmhpoQSEs5QVdXQ4hsE4VHFzOKAJHe/wcw+i2SBIlJwVmw7wJAxicxZu5szW9XkyQEdaFBVTQnl53IbqqoMTAUygesJZo5fRDBjXF1xRYqJtIxMRs1YxZ+nLadc6Tieu7ITl3Wpr6aEkq3chqr2mVkPYHxo1W6Czri3Av8ys1MJZpOLSBG1YONeBo1OZNHmfVwQX4dHL+5AzUo6dSk5C2cCYKaZlSTojvsl0N/dN5jZFQRXVV0b4RpFJAJS0jL48+fLGTVjFdUqlOa167vQt4OaEkruwpoA6O4XhBbnm9mfzMzcfZuZ3QIcilx5IhIJP6zZxeDRiazacZArT2nAsP7tqFK+VLTLkiIi3JnjJYDe7v65u39yZL27/xCxykQk3x04nM4zU5fwzqy1NKhajn/+uhtntFRTQsmbcDuSOcGlt5+bWQdg25EWJCJSNExfuo2h4xawae8hbjm9CX88rzUV1JRQjkO4Q1VuZpmhTz8GksysBsHluS+5u+7JIVJI7T6YyuOTFjF23kZa1KrI6DtP45TGVXN/okgOjufPjQ3ufjH873Ld6ehmTiKFjrszZcEWHp6wgD3JadzbpwX39GlBmZLqLyUnJpwJgLWAdtk9Frpcd1a+VyUiJ2TbvhQemrCATxZuJb5+Fd65tTvt6lWOdllSTITTqexdoEaWzz3rg+5+d3ZPMrM3zGyWmQ3L4fGmZjbJzGaa2XM5rcuy/Suh7rwikgN354Mf1nPO818xfel2hvRrw7jfnKbQkHyVa3C4+3nuPhroZGZfA83N7CMzuzXUeuQXzOwyIM7dewDNzKxlNpuNBB539zOABmbWK4d1mNkZQB13//g49lEkJqzflcwNb3zPoDGJtKlbmSn3ncGdZzWnpDrZSj7LyzmOxNAbOmZWF7gD+MbMLs7mCqtewAeh5U+BnsDyo7ZpBcwLLW8DqmS3zsxKAa8Dk83sEnefkIeaRYq9jEzn7W/X8KdPlhJXwhhxaQeu7dZITQklYsKdxxFHlqMTd98MDDezT4APzay3u2dmeUoFYGNoeRfQJZsvOxp4xMxmA32BB4D4bNbdCCwCngHuNbNG7v7XbGocCAwEaNRIDXslNizfup/BYxKZt24PvVrX5MkB8dQ7qVy0y5JiLtxjWAd+ce9xd58F3HBUaAAcAI789FbM7nXcfQQwBbgNeNvdD2S3DjgZGOXuWwjOt/TOtkD3Ue6e4O4JNWtqQpMUb2kZmfz18+X0/8vXrN5xkBev6sxbN3dVaEiBCHeoKg5YHbob4EHgoLsfAnD37JocziUYnpoNdAKW5vB15xPcz+OaY6xbATQLLScAa8OsWaRYStywh0GjE1myZT8XdqzL8IvbU6OimhJKwQk3OJoBYwiaHJYFyppZeaA6sMbdbz5q+/HATDOrB/QDrjazEe5+9BVW9wPPu3vyMda9AbxpZlcDpYArwqxZpFhJScvghc+W8frMVdSoWIZRN5zCee3rRLssiUF5OTn+g7vfC2Bm8cDCUOfcZWZW3d13HtkwNL+jF3Au8ExomOnHo7+guz+S2zp33w9cmYc6RYqd2at2MmRMImt2JnNNt4YM6deWKuXUlFCiI5wJgKMJhqfizKwi8B6QDtwNbAFGhD7/GXffzU9XVonIcdifksbTU5bw3nfraFStPP+6rTuntaiR+xNFIiicI45rCI4c7gKeA/7P3VcfedDd34lQbSIx7csl23hwXBJb96VwW8+m/P68VpQvraaEEn3h3MgpjWAOxalAa+CGo24nmezuz0aoPpGYs+tgKo99vJDx8zfRslZFXrnrNE5upKaEUniEM1TV2N3XAhcCvwNeBO4juMT2eYIjERE5Qe7OxMTNDP9oIXsPpXHf2S35Te/makoohU44x723m9kpwB53/8rMdrv7DAAz2+Pu30W2RJHib8veFIaNX8C0xVvp1KAK793enTZ11F9KCqdwhqqGAZjZLjP7DIg3s08BAzqa2VR37xvhOkWKJXfn3z+s58lJi0nLzGToBW25tWdT4tQuRAqxvJxpq5HNDPEjt5UVkTxau/MgQ8YkMWvVTk5tVo2nL+tIkxoVol2WSK7CDo7QnI0+wAx3TzezK9x9dHZhIiI5y8h03vpmNc9+upRSJUrw5IB4ru7aUE0JpcgI5+T4OGCku88muBz3ITMrQ9Add3SE6xMpVpZu2c+gMYn8uH4PZ7epxYgBHahbRf2lpGgJ54ijPnBt6IZM4wh6T8UBlczsLKCUu0+LYI0iRV5qeiavTF/By1+uoFLZUvz56s5c3KkeR13aLlIkhBMce4CPgZdD/3YLPa8y0JXgslwFh0gO5q/fw+DRiSzdup9LOtfj4QvbUV1NCaUIO2ZwhO7DcTLQI/TvVIIAyQTqauKfSM4OpWbw/GdLeePr1dSqVJY3bkrg7La1o12WyAnL7YijKnALwdBUQ4LeU5OB84F/RLQykSLs25U7GDImiXW7krm2eyOG9GtD5bJqSijFQ27BcT7QHJhBMER1OcH9MXD3fwGYWWl3T41kkSJFxb6UNJ6avIT3v19H4+rlef/2U+nRvHq0yxLJV7kFxyzgVYIbKEFwa9ePCe61cQPBJMA44IKIVShSRExbtJWh45PYvv8wA89sxu/OaUW50moXIsVPbpP3NhDcNvZFIAVIAs4OPTbJ3S9yd4WGxLSdBw5z7/v/5bZ35lC1fGnG/eZ0HrygrUJDiq1jHnG4e6qZ3QzMA64Dzg31q+oL/N7MKobuCy4Sc9ydj37cxPCPFnLgcDq/P7cVd57VnNIl1UxBirdwLsc9RHCeY4m7TzazHsAPQIZCQ2LVpj2HGDZ+AV8s2UbnhifxzBUdaVW7UrTLEikQ4fxp9FfgIuA/oc+fIhi+6hWhmkQKrcxM573v1nLeCzOYtXInD13YjjF3nabQkJgSzhHHJncfbmY9zewNoA3wLkFn3PeByu7eP6JVihQCq3ccZMiYRL5bvYvTW1TnqQEdaVS9fLTLEilw4QRHOzP7CPiQoOXIecCtwAfufk0kixMpDNIzMnnj69U8/9kySpcswcjL4/lVQkO1C5GYldvM8RLABHe/zcwGAl8Bw4B7gXUFUJ9IVC3evI/BYxJJ3LCXc9vVZsSlHahduWy0yxKJqtyuqso0s5Fm1hi4EbiHYALgycAPZjYWWO7ugyNfqkjBOZyewctfrOCV6SupUq4UL117Mv3j6+ooQ4TwhqpaAk8C9YAFwMrQuuVAOjA+YtWJRMG8dbsZPDqR5dsOcNnJ9XnownZUrVA62mWJFBq5DVVdA/yW4Eqqe4FVwJcEM8g/BJIJgkSkyEtOTefZT5bx1rerqVu5LG/d0pXerWtFuyyRQie3oar3zWw98Fho2zjgSoJW68NDm70DjI1gjSIR9/XyHTwwLpH1uw5xw6mNGdS3NZXUlFAkW7kOVbn712Z2IXCbu/8l62NmVhZoFaniRCJt76E0npi0iA/mbKBpjQr8Z+CpdG+mpoQixxLWPcfdPRn4i5n1dvcvs6xPARIjVZxIJH2ycAsPjV/AzoOp3HlWc/7vnJaULaX+UiK5Ceee468BI9x9AzDEzI4+p1HG3ZdHpDqRCNi+/zDDP1rIpKTNtK1bmTdu6kp8gyrRLkukyAjniGM5wX3HNxCc43joqMfLEFyqK1KouTvj/ruRxyYuIvlwBvef35qBZzajVJyaEorkRbjBUTe0nOzut0ewHpGI2LjnEEPHJTF96Xa6NAqaEraopf5SIscjnOBYBbxgZrcDcWY2FUgFdgPj3X1cJAsUORFHmhI+PWUJDgy/qB039GhCXAlN5BM5XuEER32gP3AS0AlII7jzXxJwl5ndBVwcOlEuUmis3H6AIWMS+WHNbs5oWYMnB8TTsJqaEoqcqHCC42yCNiMdgL7AfOBed+8APG5mTRUaUpikZ2QyauYqXpy2nLIlS/CnKzpyxSkN1C5EJJ+Ec1bwB+AMgpYjC4HpwAEzKwng7quze5KZvWFms8xsWA6PNzWzSWY208yey2ldlu1rm9l/w94ziUkLN+3l0le+4ZmpS+nTuhbT/nAWV6qTrUi+CueIYy4wFMgEMgiGq1YBU80sDvjQ3V/J+gQzuwyIc/ceZvammbXM5pLdkcDj7j7bzP5jZr2A3xy9zt2nh7Z/Fih3nPspxVxKWgZ//WI5r321iqrlS/PqdV3oF1839yeKSJ6FM3N8FfDr7B4zszJAj2we6gV8EFr+FOhJcHVWVq0I7mUOsA2oksM6zKwPcBDYklu9EnvmrNnF4DGJrNx+kMu7NOChC9tyUnk1JRSJlDxdwG5mPwsQdz+c5YggqwrAxtDyLqB2NtuMBh4xs4sIzp18nt06MytNMHdkSC61DTSzOWY2Z/v27XnYKymqDh5OZ/hHC7nyb7NIScvk7Vu78dyvOik0RCIsrJYjWVwJvBHGdgf4aVipItkElLuPMLOewP3A2+5+APjFOjN7GHjF3fcca5za3UcBowASEhI8D/skRdCMZdt5YGwSm/Ye4sZTG3N/3zZULJPXH2cROR55/U3LDHO7uQTDU7MJzokszWG7+UAj4JpjrDsH6GNmdwOdzezv7n5bHuuWYmJPciojJi1m9NwNNKtZgQ/v6EFCk2rRLkskpoTTq+ob4DDB3I14M/sitOyhf8u6+9HnOcYDM82sHtAPuNrMRrj70VdY3Q88H2qimO06dz8zSy3TFRqxa0rSZh6asJDdyanc3bs59/ZRU0KRaDD38Ed1zGyKu/cLc9uqwLnADHcv0JPaCQkJPmfOnIJ8SYmgbftTeGTCQqYs2EL7epV55oqOtK+npoQi+c3M5rp7Qm7b5XWo6n9/3plZY3dfm9OG7r6bn66sEskzd2f03A2MmLSYQ2kZDOrbmtvPUFNCkWgLKzjMrIW7rwA+zrL6GTOb5u6vR6Y0iWXrdyXz4LgkZi7fQdcmVXn68o40r1kx2mWJCOGd4+gJPBe6C+DFocl9TnD08aqZJbv7exGuU2JEZqbzzqw1PPPJUgx4/JL2XNe9MSXUlFCk0AjniKM70M/dd5lZprv3PvKAmVUk/CutRI5pxbb9DB6TxNy1uzmrVU2eGNCBBlXVlFCksAln5njWnlGTIbjXeKixYbK7KzjkhKRlZDJqxir+PG055cvE8fyvOjHg5PrqLyVSSIUzVDWC4P4bRnAXQIB/mtlQ4FEze8zdF0ewRinGFmzcy/2jE1m8eR/94+sy/OL21KxUJtplicgxhDNUdQEwDtgMPBj6K7AzwWzw0goNOR4paRm8OG05r89cRbUKpXnt+lPo26FOtMsSkTCEExx7CWZ+rw0tdwC2A1WBRyJXmhRX36/exZAxiazacZBfJTRg6AXtqFK+VLTLEpEwhRMcdYCzgB3AHoIhq1SgC5BuZvvcfV3kSpTi4sDhdEZOWcI/Z6+lQdVyvPvr7vRsWSPaZYlIHoUTHE5wH44MIB7YSdD9NgMoAzwG3Byh+qSY+HLpNoaOTWLzvhRuPb0pfzy/FeVLqymhSFEUzm/uVuBrgqGqmwiCpCFQCZgGvBWx6qTI230wlccnLmLsfzfSolZFRt95Gqc0rhrtskTkBIQTHFUJzmvUBNYBq4E0ggAZ4e79I1eeFFXuzuSkLTzy0QL2JKfx2z4tuLtPC8qUVFNCkaIunOAYS3BOozLwD2AicBXB7WOrmdnZ7v55xCqUImfrvhQeGr+ATxdtJb5+Fd65tTvt6lWOdlkikk/CmQD4WDarX4H/tTlPyfeqpEhydz6Ys54RkxaTmp7JA/3a8OueTSmppoQixcoJnZ1UaMgR63Ym88C4RL5ZsZNuTasx8vKONK1RIdpliUgE6LIWOSEZmc4/vl3Ds58sJa6EMeLSDlzbrZGaEooUYwoOOW7Lt+5n0JhE/rtuD71b1+SJAfHUO6lc7k8UkSJNwSF5lpqeyWtfreSlL1ZQoUwcL17VmUs611NTQpEYoeCQPPlx/R4Gj0lkyZb9XNSpHo9c1I4aFdWUUCSWKDgkLIdSM3hx2jJen7mKmpXK8PqNCZzbrna0yxKRKFBwSK5mr9rJkDGJrNmZzDXdGjKkX1uqlFNTQpFYpeCQHO1PSePpKUt477t1NKpWnn/d1p3TWqgpoUisU3BItr5YspWh4xawdV8Kt/Vsyh/Oa0250moXIiIKDjnKzgOHeWziIibM30Sr2hV55brTOLmRmhKKyE8UHAIE7UI+TtzM8I8Wsj8ljfvObsndvVtQuqTahYjIzyk4hC17Uxg2Polpi7fRqUEVRl7RnTZ11JRQRLKn4Ihh7s6/f1jPk5MWk5aZybD+bbnl9KbEqV2IiByDgiNGrd15kCFjkpi1aic9mlXn6cvjaVxdTQlFJHcKjhiTkem8+fVqnvtsKaVKlOCpy+K5umtDtQsRkbApOGLI0i37GTT6R37csJdz2tZixKXx1KlSNtpliUgRo+CIAanpmbz85Qpemb6CSmVL8ZdrTuaijnV1lCEix0XBUczNX7+HQaN/ZNnWA1zauR4PX9SeahVKR7ssESnCFBzF1KHUDJ77dClvfrOa2pXL8uazxTn8AAAN10lEQVTNCfRpo6aEInLiFBzF0LcrdjBkbBLrdiVzXfdGDOnXhkpl1ZRQRPJHxILDzN4A2gGT3H1ENo83BV4CKgPfu/sfclhXBfg3EAccBK5y99RI1V2U7T2UxlOTF/PvH9bTpHp5/j3wVE5tVj3aZYlIMRORfhJmdhkQ5+49gGZm1jKbzUYCj7v7GUADM+uVw7rrgOfd/TxgC9A3EjUXdZ8t2sp5L3zFB3PWc8eZzZhy35kKDRGJiEgdcfQCPggtfwr0BJYftU0rYF5oeRtQJbt17v5KlufUDK3/BTMbCAwEaNSo0YlVX4TsOHCY4R8tZGLiZtrUqcTrNybQscFJ0S5LRIqxSAVHBWBjaHkX0CWbbUYDj5jZbIKjiAeA+GzWAWBmPYCq7j47uxd091HAKICEhATPp/0otNydCfM38ejHCzlwOJ3fn9uKO89qrqaEIhJxkQqOA0C50HJFshkSc/cRZtYTuB94290PANmtw8yqAX8FLo9QvUXKpj2HGDZ+AV8s2cbJjU7imcs70rJ2pWiXJSIxIlLBMZdgeGo20AlYmsN284FGwDU5rTOz0sCHwAPuvjZC9RYJmZnOv75fx9NTlpCR6Tx8YTtuOq2JmhKKSIGKVHCMB2aaWT2gH3C1mY1w92FHbXc/wYnv5GOs+zXBUNdQMxsKvOru/4lQ3YXW6h0HGTwmke9X7+L0FtV5akBHGlUvH+2yRCQGmXtkTgeYWVXgXGCGu2+JyIvkICEhwefMmVOQLxkx6RmZ/P3r1bzw2TJKlyzBQ/3bcWVCA7ULEZF8Z2Zz3T0ht+0iNo/D3Xfz05VVchwWbdrH4DGJJG3cy7ntajPi0g7UrqymhCISXZo5XggdTs/gpS9W8Or0lZxUvhQvX9uFC+Lr6ChDRAoFBUchM3ftbgaPSWTFtgNc1qU+D/VvR1U1JRSRQkTBUUgcPJzOs58u5R/frqFu5bK8dUtXereuFe2yRER+QcFRCMxcvp0HxiaxYfchbuzRmEF921CxjP5rRKRw0rtTFO1NTuOJyYv4YM4GmtaowAd39KBb02rRLktE5JgUHFEydcEWHpqwgF0HU7mrV3PuO7slZUvFRbssEZFcKTgK2Pb9QVPCSUmbaVe3Mm/d3JUO9atEuywRkbApOAqIuzN23kYem7iIQ6kZ3H9+awae2YxScWpKKCJFi4KjAGzYncyD4xYwY9l2TmlclZGXd6RFrYrRLktE5LgoOCIoM9N597u1jJyyBAeGX9SOG3s0oYSaEopIEabgiJCV2w8wZEwiP6zZzRkta/DkgHgaVlNTQhEp+hQc+SwtI5PXZ67ixWnLKVcqjmev7MTlXeqrXYiIFBsKjny0YONeBo9JZOGmffTrUIdHL2lPrUpqSigixYuCIx+kpGXw1y+W89pXq6havjSvXteFfvF1o12WiEhEKDhO0Jw1uxg0JpFV2w9yxSkNGNa/LSeVV1NCESm+FBzH6cDhdP40dQnvzF5LvSrleOfWbpzZqma0yxIRiTgFx3H4atl2HhybxKa9h7ipRxPuP781FdSUUERihN7t8mBPciqPT1zMmHkbaF6zAh/e0YOEJmpKKCKxRcERpilJm3lowkJ2J6dyT+8W3NOnhZoSikhMUnDkYtu+FB6esJCpC7fQvl5l3r61K+3rqSmhiMQuBUcO3J0P525gxMRFpKRnMrhvG24/oykl1ZRQRGKcgiMb63cl8+C4JGYu30HXJlV5+vKONK+ppoQiIqDg+JmMTOedWWv40ydLMeDxS9pzXffGakooIpKFgiOLpyYv5u9fr+asVjV58rJ46p9ULtoliYgUOgqOLG7s0YR29Soz4GQ1JRQRyYmCI4tG1cvTqLpan4uIHIsuERIRkTxRcIiISJ4oOEREJE8UHCIikicKDhERyRMFh4iI5ImCQ0RE8kTBISIieWLuHu0a8p2ZbQfWHufTawA78rGcokD7HBu0z7HhRPa5sbvneg/sYhkcJ8LM5rh7QrTrKEja59igfY4NBbHPGqoSEZE8UXCIiEieKDh+aVS0C4gC7XNs0D7Hhojvs85xiIhInuiIQ0RE8kTBEWPMrLaZzTzG443MbLqZfWFmo6yI39Eqt/3Nsl0HM/usIGqKtDzs88dm1rkgaoq0MH6um5nZ52Y238weKsjaiqOYDo4wfthKhX65vjGzWwuytkgws6rA20CFY2x2B3CXu/cBGgLxBVFbJIS5v4TC8XmgVEHUFUl52OfrgJXuPr9ACougMPf5HuBhd+8MnG9muc5VKKzMrIqZTTGzT81snJmVzmG7N8xslpkNy+8aYjY4wvxhuxeY6+6nA1eYWaUCKS5yMoCrgH05beDuQ919cejT6hTtyVO57m/ILcCXkS+nQOS6z2ZWDXgO2G1mvQuqsAgK5/95J9DRzGoDZYA9BVFYhFwHPO/u5wFbgL5Hb2BmlwFx7t4DaGZmLfOzgJgNDsL7YesFfBBangEU6YlE7r7P3feGs62ZXQUsdPdNES4rYsLZXzOrDlwPPFswVUVWmP/HvwM+BP4G3GhmF0e+ssgJc5+nAqcCvwW+ANIjXliEuPsr7n5kWLUmsC2bzXrx03vXp0DP/KwhZoMjzB+2CsDG0PIuoHZkqyoczKwZ8Efg/6JdSwF4GnjA3dOiXUgBOhl42d23ELy59IpuOQViCHCzuw8FygHnRrmeE2ZmPYCq7j47m4cj+t4Vs8ERpgMEP2QAFYmB71doCO994NZwj06KuLOAkWY2HehsZiOiXE9BWAE0Cy0ncPx93YqSpkBDMysLdAGK9DyE0HDjX4Gczr1G9L2r2L8RnqC5/HSI1wlYE71S8p+Z9TGze45aPQRoBPw1dHXVWVEoLSKy2193b+Xuvdy9FzDf3fP9RGI05fB//Axwj5l9A5wJvFnwlUVODvv8CDAd2A6sJxiuKpJCJ8M/JDhSzin0I/reFfMTAM1surv3MrM+QDt3fynLY42BycA04DTgVHfPiFKpIiKY2V3Ak8CPoVVfAqWy/tFjZpWBmcDnQD+C9658G0GI+eDIjZnVI0juT2Jk6EZEioHQsPO5wIzQ+az8+9oKDhERyQud4xARkTxRcIiEmFmJrC1Wjv5cRAIKDokZZnaemTUILb9sZo3N7OMsm5wLbDSz2aErjjYBT5rZC2b2bOijZJav94SZNQ0tlzazMTm87idmVv5IEIVaRcSZWVyWbczMJphZg6NbSJhZySPbmtljZtY79NpDzKxS6OvHHfWc681smZlNO+pjlZldc2LfSYl1Cg6JJWWAe80sARgA/B1ICL3x3uPunwBjgauBSwhmG38ExAEvElyZknXG8Sn8NAfiXCDZzNqEPkqbWQszexhIc/dk4HyCq1xOBj4BpmUJiUeA/xJce/9l6FLoA6H5JV8CF5hZRYJOBz2AWkBLoDFw0N0zQsF05Hc6FRjl7udk/QDeIeiaIHLcSua+iUixMZHgGv7WwAUEfX5ecvcrjtru3wRvrsuBDUBZd99gZikAob/udxBcDjkvdNRyMsGb+hCgG3AZ0A44B2hgZq8Cv3X3KWY20d0vDH2tEmZ2SWjbV4HMUG80zGx+aH4Joc/rE/QPuweYD3wbWm5hZjOAFsClwPdAJjDQzI7uY9QMGHSc3z8RQMEhMcLM+gG/B74CDgNXELxZZ5jZ5wRv+kdarFxNMPP2WWAzUDc0p2cTQOiv+3nufraZ9QJ+E9ruD+6+z8xeDr3GacCdwDPufpeZPRM62sHMVgC7gf8QNBz8huBoZICZ9XD3WVlqjyMIggygSWj71kAdoCMwlGA2+B3u/n2W3R7l7j/rwWVmw4/7mygSouCQmBD6S38x8Ki73xQ66f0FQYdk3P0fAKFz4e8SvEmvdvd0MysF/IqgWdwRnULDSCcB44F/hp53McGQ2EFgJEFXVszsV8BD7n449PlwYBIwDzDgFWCmu68ys7FA59B20wh+T+8lCJpHgRsI2sCnEUwEO4VgaGpVfn2/RI5FwSGx5sjEpb4EbRkgODd9IbAs9Pn1hI44Qj2BphJ0lG2b5evMd/dzQkccvdx9pZllmFlHgvMUBwmODh4gmED6GXCfmfUP1dAEuAh4meA+IKlZvnbK/4oNzkscKfJ0YATBuY1OBOHSlOC8DMCUo/ZVQ1USEQoOiTVxZtYJ+BPQm2DI5y2CbqK9CBrCXUfwBnsmwQntGgRHILWB1cf42ncDWwmCI5ng4pMJQCt3/3PoRLi5+8jQEcdEd59jZmUIQuzmYxXu7t+Y2QcE7cGnELS9TzOzeQTnNh476ikaqpKIUHBILHmM4B4sScA17r6doOldSwAzu4DgTX8OwdVHJYHyBH/Z9wfGmdmdoTbWJx81VMWRe5eYWXUPWjIkhT6/P/R4qpnVMbOf3TzM3Q+Hhs7s6EtxQ8+PA0qEWr+/RnA0MwxYG7ocuD3BOZUuodohGP7K6YhjcF6/cSJZKTgkJphZPFCNICSuIxiGKgGUDX3UITgHcVVo+zpAaaAScG3opPdNwUMWR3BnyPMsuCfCGaHn3ETwhn70/RFKmVkZdz/s7r8zszsIhspey7JNmdBrfQIcCJ3b2BH6twTwmZm9RnDl1SqCo452BEdLfyQ40hltZte6+0qC3+1X3P3Fo74Pw9HvvZwg9aoSyScWNJUrGTqSOdZ25sf5i2dmJY/MJQkdpZQ40rH5RL6uSF4oOEREJE80c1xERPJEwSEiInmi4BARkTxRcIiISJ4oOEREJE/+H37WiX38YjDpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 可视化 所解释的方差与选取的特征数目之间的关系\n",
    "import matplotlib.pyplot as plt\n",
    "plt.plot(np.array(range(len(compressor.explained_variance_ratio_))) + 1, \n",
    "         np.cumsum(compressor.explained_variance_ratio_))\n",
    "plt.xlabel('选取的特征数目')\n",
    "plt.ylabel('累计所解释的方差累')\n",
    "plt.show(); # LDA将原始的4个变量压缩为2个，这2个变量即能解释100%的方差"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {
    "height": "786.465px",
    "left": "40.9922px",
    "top": "113.535px",
    "width": "370.918px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
